Micron Document
<!DOCTYPE html>
<html class="client-nojs vector-feature-night-mode-disabled vector-feature-language-in-header-enabled vector-feature-language-in-main-page-header-disabled vector-feature-page-tools-pinned-disabled vector-feature-toc-pinned-clientpref-1 vector-feature-main-menu-pinned-disabled vector-feature-limited-width-clientpref-1 vector-feature-limited-width-content-enabled vector-feature-custom-font-size-clientpref-1 vector-feature-appearance-pinned-clientpref-1 vector-sticky-header-enabled" lang="en" dir="ltr"><head>
<meta charset="UTF-8">
<title>Nonlinear programming</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="canonical" href="https://en.wikipedia.org/wiki/Nonlinear_programming"> <link href="./mw/ext.cite.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/ext.math.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/skins.vector.icons.css" rel="stylesheet" type="text/css">
<link href="./mw/skins.vector.search.codex.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/skins.vector.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/user.styles.css" rel="stylesheet" type="text/css">
<meta name="ResourceLoaderDynamicStyles" content="">
<link rel="stylesheet" type="text/css" href="./mw/site.styles.css">
<link rel="stylesheet" type="text/css" href="./mw/noscript.css">
<link rel="stylesheet" type="text/css" href="./footer.css">
<link rel="stylesheet" type="text/css" href="./vector-2022.css">
</head>
<body class="skin--responsive skin-vector skin-vector-search-vue mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject page-Nonlinear_programming rootpage-Nonlinear_programming skin-vector-2022 action-view">
<div class="mw-page-container">
<div class="mw-page-container-inner">
<div class="mw-content-container">
<main id="content" class="mw-body">
<header class="mw-body-header vector-page-titlebar">
<h1 id="firstHeading" class="firstHeading mw-first-heading">
<span id="openzim-page-title" class="mw-page-title-main"><span class="mw-page-title-main">Nonlinear programming</span></span>
</h1>
</header>
<a id="top"></a>
<div id="bodyContent" class="vector-body ve-init-mw-desktopArticleTarget-targetContainer" aria-labelledby="firstHeading" data-mw-ve-target-container="">
<div id="mw-content-text" class="mw-body-content mw-content-ltr" lang="en" dir="ltr"><div class="mw-content-ltr mw-parser-output" lang="en" dir="ltr">
<p>In <a href="Mathematics" title="Mathematics">mathematics</a>, <b>nonlinear programming</b> (<b>NLP</b>) is the process of solving an <a href="Optimization_problem" title="Optimization problem">optimization problem</a> where some of the constraints are not <a href="Linear_inequality" title="Linear inequality">linear equalities</a> or the objective function is not a <a href="Linear_function" title="Linear function">linear function</a>. An <a href="Optimization_problem" title="Optimization problem">optimization problem</a> is one of calculation of the extrema (maxima, minima or stationary points) of an <a href="Objective_function" class="mw-redirect" title="Objective function">objective function</a> over a set of unknown <a href="Function_of_a_real_variable" title="Function of a real variable">real variables</a> and conditional to the satisfaction of a <a href="Simultaneous_equations" class="mw-redirect" title="Simultaneous equations">system</a> of <a href="Equation" title="Equation">equalities</a> and <a href="Inequality_(mathematics)" title="Inequality (mathematics)">inequalities</a>, collectively termed <a href="Constraint_(mathematics)" title="Constraint (mathematics)">constraints</a>. It is the sub-field of <a href="Mathematical_optimization" title="Mathematical optimization">mathematical optimization</a> that deals with problems that are not linear.
</p>
<meta property="mw:PageProp/toc">
<div class="mw-heading mw-heading2"><h2 id="Definition_and_discussion">Definition and discussion</h2></div>
<p>Let <i>n</i>, <i>m</i>, and <i>p</i> be positive integers. Let <i>X</i> be a subset of <i>R<sup>n</sup></i> (usually a box-constrained one), let <i>f</i>, <i>g<sub>i</sub></i>, and <i>h<sub>j</sub></i> be <a href="Real-valued_function" title="Real-valued function">real-valued functions</a> on <i>X</i> for each <i>i</i> in {<i>1</i>, ..., <i>m</i>} and each <i>j</i> in {<i>1</i>, ..., <i>p</i>}, with at least one of <i>f</i>, <i>g<sub>i</sub></i>, and <i>h<sub>j</sub></i> being nonlinear.
</p><p>A nonlinear programming problem is an <a href="Optimization_problem" title="Optimization problem">optimization problem</a> of the form
</p>
<dl><dd><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\begin{aligned}{\text{minimize }}&amp;f(x)\\{\text{subject to }}&amp;g_{i}(x)\leq 0{\text{ for each }}i\in \{1,\dotsc ,m\}\\&amp;h_{j}(x)=0{\text{ for each }}j\in \{1,\dotsc ,p\}\\&amp;x\in X.\end{aligned}}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mtable columnalign="right left right left right left right left right left right left" rowspacing="3pt" columnspacing="0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em" displaystyle="true">
<mtr>
<mtd>
<mrow class="MJX-TeXAtom-ORD">
<mtext>minimize&nbsp;</mtext>
</mrow>
</mtd>
<mtd>
<mi>f</mi>
<mo stretchy="false">(</mo>
<mi>x</mi>
<mo stretchy="false">)</mo>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow class="MJX-TeXAtom-ORD">
<mtext>subject to&nbsp;</mtext>
</mrow>
</mtd>
<mtd>
<msub>
<mi>g</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<mo stretchy="false">(</mo>
<mi>x</mi>
<mo stretchy="false">)</mo>
<mo>≤<!-- ≤ --></mo>
<mn>0</mn>
<mrow class="MJX-TeXAtom-ORD">
<mtext>&nbsp;for each&nbsp;</mtext>
</mrow>
<mi>i</mi>
<mo>∈<!-- ∈ --></mo>
<mo fence="false" stretchy="false">{</mo>
<mn>1</mn>
<mo>,</mo>
<mo>…<!-- … --></mo>
<mo>,</mo>
<mi>m</mi>
<mo fence="false" stretchy="false">}</mo>
</mtd>
</mtr>
<mtr>
<mtd></mtd>
<mtd>
<msub>
<mi>h</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>j</mi>
</mrow>
</msub>
<mo stretchy="false">(</mo>
<mi>x</mi>
<mo stretchy="false">)</mo>
<mo>=</mo>
<mn>0</mn>
<mrow class="MJX-TeXAtom-ORD">
<mtext>&nbsp;for each&nbsp;</mtext>
</mrow>
<mi>j</mi>
<mo>∈<!-- ∈ --></mo>
<mo fence="false" stretchy="false">{</mo>
<mn>1</mn>
<mo>,</mo>
<mo>…<!-- … --></mo>
<mo>,</mo>
<mi>p</mi>
<mo fence="false" stretchy="false">}</mo>
</mtd>
</mtr>
<mtr>
<mtd></mtd>
<mtd>
<mi>x</mi>
<mo>∈<!-- ∈ --></mo>
<mi>X</mi>
<mo>.</mo>
</mtd>
</mtr>
</mtable>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}{\text{minimize }}&amp;f(x)\\{\text{subject to }}&amp;g_{i}(x)\leq 0{\text{ for each }}i\in \{1,\dotsc ,m\}\\&amp;h_{j}(x)=0{\text{ for each }}j\in \{1,\dotsc ,p\}\\&amp;x\in X.\end{aligned}}}</annotation>
</semantics>
</math></span><img src="./7953a3185f4689d7b3a4d271b5c990207aa2c148.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -5.671ex; width:43.868ex; height:12.343ex;" alt="{\displaystyle {\begin{aligned}{\text{minimize }}&amp;f(x)\\{\text{subject to }}&amp;g_{i}(x)\leq 0{\text{ for each }}i\in \{1,\dotsc ,m\}\\&amp;h_{j}(x)=0{\text{ for each }}j\in \{1,\dotsc ,p\}\\&amp;x\in X.\end{aligned}}}" loading="lazy"></span></dd></dl>
<p>Depending on the constraint set, there are several possibilities:
</p>
<ul><li><i>feasible</i> problem is one for which there exists at least one set of values for the choice variables satisfying all the constraints.</li>
<li>an <i>infeasible</i> problem is one for which no set of values for the choice variables satisfies all the constraints. That is, the constraints are mutually contradictory, and no solution exists; the feasible set is the <a href="Empty_set" title="Empty set">empty set</a>.</li>
<li><i>unbounded</i> problem is a feasible problem for which the objective function can be made to be better than any given finite value. Thus there is no optimal solution, because there is always a feasible solution that gives a better objective function value than does any given proposed solution.</li></ul>
<p>Most realistic applications feature feasible problems, with infeasible or unbounded problems seen as a failure of an underlying model. In some cases, infeasible problems are handled by minimizing a sum of feasibility violations.
</p><p>Some special cases of nonlinear programming have specialized solution methods:
</p>
<ul><li>If the objective function is <a href="Concave_function" title="Concave function">concave</a> (maximization problem), or <a href="Convex_function" title="Convex function">convex</a> (minimization problem) and the constraint set is <a href="Convex_set" title="Convex set">convex</a>, then the program is called convex and general methods from <a href="Convex_optimization" title="Convex optimization">convex optimization</a> can be used in most cases.</li>
<li>If the objective function is <a href="Quadratic_function" title="Quadratic function">quadratic</a> and the constraints are linear, <a href="Quadratic_programming" title="Quadratic programming">quadratic programming</a> techniques are used.</li>
<li>If the objective function is a ratio of a concave and a convex function (in the maximization case) and the constraints are convex, then the problem can be transformed to a convex optimization problem using <a href="Fractional_programming" title="Fractional programming">fractional programming</a> techniques.</li></ul>
<div class="mw-heading mw-heading2"><h2 id="Applicability">Applicability</h2></div>
<p>A typical non-<a href="Convex_optimization" title="Convex optimization">convex</a> problem is that of optimizing transportation costs by selection from a set of transportation methods, one or more of which exhibit <a href="Economy_of_scale" class="mw-redirect" title="Economy of scale">economies of scale</a>, with various connectivities and capacity constraints. An example would be petroleum product transport given a selection or combination of pipeline, rail tanker, road tanker, river barge, or coastal tankship. Owing to economic batch size the cost functions may have discontinuities in addition to smooth changes.
</p><p>In experimental science, some simple data analysis (such as fitting a spectrum with a sum of peaks of known location and shape but unknown magnitude) can be done with linear methods, but in general these problems are also nonlinear. Typically, one has a theoretical model of the system under study with variable parameters in it and a model the experiment or experiments, which may also have unknown parameters. One tries to find a best fit numerically. In this case one often wants a measure of the precision of the result, as well as the best fit itself.
</p>
<div class="mw-heading mw-heading2"><h2 id="Methods_for_solving_a_general_nonlinear_program">Methods for solving a general nonlinear program</h2></div>
<div class="mw-heading mw-heading3"><h3 id="Analytic_methods">Analytic methods</h3></div>
<p>Under <a href="Differentiability" class="mw-redirect" title="Differentiability">differentiability</a> and <a href="Constraint_qualification" class="mw-redirect" title="Constraint qualification">constraint qualifications</a>, the <a href="Karush%E2%80%93Kuhn%E2%80%93Tucker_conditions" title="Karush–Kuhn–Tucker conditions">Karush–Kuhn–Tucker (KKT) conditions</a> provide necessary conditions for a solution to be optimal. If some of the functions are non-differentiable, <a href="Subderivative" title="Subderivative">subdifferential</a> versions of <a href="Karush%E2%80%93Kuhn%E2%80%93Tucker_conditions" title="Karush–Kuhn–Tucker conditions">Karush–Kuhn–Tucker (KKT) conditions</a> are available.<sup id="cite_ref-1" class="reference"><a href="#cite_note-1"><span class="cite-bracket">[</span>1<span class="cite-bracket">]</span></a></sup>
</p><p>Under convexity, the KKT conditions are sufficient for a <a href="Global_optimum" class="mw-redirect" title="Global optimum">global optimum</a>. Without convexity, these conditions are sufficient only for a <a href="Local_optimum" class="mw-redirect" title="Local optimum">local optimum</a>. In some cases, the number of local optima is small, and one can find all of them analytically and find the one for which the objective value is smallest.<sup id="cite_ref-:0_2-0" class="reference"><a href="#cite_note-:0-2"><span class="cite-bracket">[</span>2<span class="cite-bracket">]</span></a></sup>
</p>
<div class="mw-heading mw-heading3"><h3 id="Numeric_methods">Numeric methods</h3></div>
<p>In most realistic cases, it is very hard to solve the KKT conditions analytically, and so the problems are solved using <a href="Numerical_analysis" title="Numerical analysis">numerical methods</a>. These methods are iterative: they start with an initial point, and then proceed to points that are supposed to be closer to the optimal point, using some update rule. There are three kinds of update rules:<sup id="cite_ref-:0_2-1" class="reference"><a href="#cite_note-:0-2"><span class="cite-bracket">[</span>2<span class="cite-bracket">]</span></a></sup><sup class="reference nowrap"><span title="Location: 5.1.2">: 5.1.2 </span></sup>
</p>
<ul><li>Zero-order routines - use only the values of the objective function and constraint functions at the current point;</li>
<li>First-order routines - use also the values of the <i><a href="Gradient" title="Gradient">gradients</a></i> of these functions;</li>
<li>Second-order routines - use also the values of the <a href="Hessian_matrix" title="Hessian matrix"><i>Hessians</i></a> of these functions.</li></ul>
<p>Third-order routines (and higher) are theoretically possible, but not used in practice, due to the higher computational load and little theoretical benefit.
</p>
<div class="mw-heading mw-heading3"><h3 id="Branch_and_bound">Branch and bound</h3></div>
<p>Another method involves the use of <a href="Branch_and_bound" title="Branch and bound">branch and bound</a> techniques, where the program is divided into subclasses to be solved with convex (minimization problem) or linear approximations that form a lower bound on the overall cost within the subdivision. With subsequent divisions, at some point an actual solution will be obtained whose cost is equal to the best lower bound obtained for any of the approximate solutions. This solution is optimal, although possibly not unique. The algorithm may also be stopped early, with the assurance that the best possible solution is within a tolerance from the best point found; such points are called ε-optimal. Terminating to ε-optimal points is typically necessary to ensure finite termination. This is especially useful for large, difficult problems and problems with uncertain costs or values where the uncertainty can be estimated with an appropriate reliability estimation.
</p>
<div class="mw-heading mw-heading2"><h2 id="Implementations">Implementations</h2></div>
<p>There exist numerous nonlinear programming solvers, including open source:
</p>
<ul><li><a href="ALGLIB" title="ALGLIB">ALGLIB</a> (C++, C#, Java, Python API) implements several first-order and derivative-free nonlinear programming solvers</li>
<li><a rel="nofollow" class="external text" href="https://nlopt.readthedocs.io/en/latest/">NLopt</a> (C/C++ implementation, with numerous interfaces including Julia, Python, R, MATLAB/Octave), includes various nonlinear programming solvers</li>
<li><a href="SciPy" title="SciPy">SciPy</a> (de facto standard for scientific Python) has scipy.optimize solver, which includes several nonlinear programming algorithms (zero-order, first order and second order ones).</li>
<li><a href="IPOPT" title="IPOPT">IPOPT</a> (C++ implementation, with numerous interfaces including C, Fortran, Java, AMPL, R, Python, etc.) is an <a href="Interior_point_method" class="mw-redirect" title="Interior point method">interior point method</a> solver (zero-order, and optionally first order and second order derivatives).</li></ul>
<div class="mw-heading mw-heading2"><h2 id="Numerical_Examples">Numerical Examples</h2></div>
<div class="mw-heading mw-heading3"><h3 id="2-dimensional_example">2-dimensional example</h3></div>

<p>A simple problem (shown in the diagram) can be defined by the constraints
<span class="mwe-math-element mwe-math-element-block"><span class="mwe-math-mathml-display mwe-math-mathml-a11y" style="display: none;"><math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\begin{aligned}x_{1}&amp;\geq 0\\x_{2}&amp;\geq 0\\x_{1}^{2}+x_{2}^{2}&amp;\geq 1\\x_{1}^{2}+x_{2}^{2}&amp;\leq 2\end{aligned}}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mtable columnalign="right left right left right left right left right left right left" rowspacing="3pt" columnspacing="0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em" displaystyle="true">
<mtr>
<mtd>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
</msub>
</mtd>
<mtd>
<mi></mi>
<mo>≥<!-- ≥ --></mo>
<mn>0</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msub>
</mtd>
<mtd>
<mi></mi>
<mo>≥<!-- ≥ --></mo>
<mn>0</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<msubsup>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo>+</mo>
<msubsup>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
</mtd>
<mtd>
<mi></mi>
<mo>≥<!-- ≥ --></mo>
<mn>1</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<msubsup>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo>+</mo>
<msubsup>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
</mtd>
<mtd>
<mi></mi>
<mo>≤<!-- ≤ --></mo>
<mn>2</mn>
</mtd>
</mtr>
</mtable>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}x_{1}&amp;\geq 0\\x_{2}&amp;\geq 0\\x_{1}^{2}+x_{2}^{2}&amp;\geq 1\\x_{1}^{2}+x_{2}^{2}&amp;\leq 2\end{aligned}}}</annotation>
</semantics>
</math></span></span>
with an objective function to be maximized
<span class="mwe-math-element mwe-math-element-block"><span class="mwe-math-mathml-display mwe-math-mathml-a11y" style="display: none;"><math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f(\mathbf {x} )=x_{1}+x_{2}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>f</mi>
<mo stretchy="false">(</mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">x</mi>
</mrow>
<mo stretchy="false">)</mo>
<mo>=</mo>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
</msub>
<mo>+</mo>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle f(\mathbf {x} )=x_{1}+x_{2}}</annotation>
</semantics>
</math></span></span>
where <span class="texhtml"><b>x</b> = (<i>x</i><sub>1</sub>, <i>x</i><sub>2</sub>)</span>.
</p>
<div class="mw-heading mw-heading3"><h3 id="3-dimensional_example">3-dimensional example</h3></div>

<p>Another simple problem (see diagram) can be defined by the constraints
<span class="mwe-math-element mwe-math-element-block"><span class="mwe-math-mathml-display mwe-math-mathml-a11y" style="display: none;"><math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\begin{aligned}x_{1}^{2}-x_{2}^{2}+x_{3}^{2}&amp;\leq 2\\x_{1}^{2}+x_{2}^{2}+x_{3}^{2}&amp;\leq 10\end{aligned}}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mtable columnalign="right left right left right left right left right left right left" rowspacing="3pt" columnspacing="0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em" displaystyle="true">
<mtr>
<mtd>
<msubsup>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo>−<!-- − --></mo>
<msubsup>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo>+</mo>
<msubsup>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>3</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
</mtd>
<mtd>
<mi></mi>
<mo>≤<!-- ≤ --></mo>
<mn>2</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<msubsup>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo>+</mo>
<msubsup>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo>+</mo>
<msubsup>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>3</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
</mtd>
<mtd>
<mi></mi>
<mo>≤<!-- ≤ --></mo>
<mn>10</mn>
</mtd>
</mtr>
</mtable>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}x_{1}^{2}-x_{2}^{2}+x_{3}^{2}&amp;\leq 2\\x_{1}^{2}+x_{2}^{2}+x_{3}^{2}&amp;\leq 10\end{aligned}}}</annotation>
</semantics>
</math></span></span>
with an objective function to be maximized
<span class="mwe-math-element mwe-math-element-block"><span class="mwe-math-mathml-display mwe-math-mathml-a11y" style="display: none;"><math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f(\mathbf {x} )=x_{1}x_{2}+x_{2}x_{3}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>f</mi>
<mo stretchy="false">(</mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">x</mi>
</mrow>
<mo stretchy="false">)</mo>
<mo>=</mo>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
</msub>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msub>
<mo>+</mo>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msub>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>3</mn>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle f(\mathbf {x} )=x_{1}x_{2}+x_{2}x_{3}}</annotation>
</semantics>
</math></span></span>
where <span class="texhtml"><b>x</b> = (<i>x</i><sub>1</sub>, <i>x</i><sub>2</sub>, <i>x</i><sub>3</sub>)</span>.
</p>
<div class="mw-heading mw-heading2"><h2 id="See_also">See also</h2></div>
<ul><li><a href="Curve_fitting" title="Curve fitting">Curve fitting</a></li>
<li><a href="Least_squares" title="Least squares">Least squares minimization</a></li>
<li><a href="Linear_programming" title="Linear programming">Linear programming</a></li>
<li><a href="Nl_(format)" title="Nl (format)">nl (format)</a></li>
<li><a href="Nonlinear_least_squares" class="mw-redirect" title="Nonlinear least squares">Nonlinear least squares</a></li>
<li><a href="List_of_optimization_software" title="List of optimization software">List of optimization software</a></li>
<li><a href="Quadratically_constrained_quadratic_programming" class="mw-redirect" title="Quadratically constrained quadratic programming">Quadratically constrained quadratic programming</a></li>
<li><a href="Werner_Fenchel" title="Werner Fenchel">Werner Fenchel</a>, who created the foundation for nonlinear programming</li></ul>
<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>
<style data-mw-deduplicate="TemplateStyles:r1239543626">
/* start https://en.wikipedia.org/ */


.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}


/* end https://en.wikipedia.org/ */
</style><div class="reflist">
<div class="mw-references-wrap"><ol class="references">
<li id="cite_note-1"><span class="mw-cite-backlink"><b><a href="#cite_ref-1">^</a></b></span> <span class="reference-text">
<style data-mw-deduplicate="TemplateStyles:r1238218222">
/* start https://en.wikipedia.org/ */


.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}


/* end https://en.wikipedia.org/ */
</style><cite id="CITEREFRuszczyński2006" class="citation book cs1"><a href="Andrzej_Piotr_Ruszczy%C5%84ski" title="Andrzej Piotr Ruszczyński">Ruszczyński, Andrzej</a> (2006). <i>Nonlinear Optimization</i>. Princeton, NJ: <a href="Princeton_University_Press" title="Princeton University Press">Princeton University Press</a>. pp.&nbsp;xii+454. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a>&nbsp;<bdi>978-0691119151</bdi>. <a href="MR_(identifier)" class="mw-redirect" title="MR (identifier)">MR</a>&nbsp;<a rel="nofollow" class="external text" href="https://mathscinet.ams.org/mathscinet-getitem?mr=2199043">2199043</a>.</cite></span>
</li>
<li id="cite_note-:0-2"><span class="mw-cite-backlink">^ <a href="#cite_ref-:0_2-0"><sup><i><b>a</b></i></sup></a> <a href="#cite_ref-:0_2-1"><sup><i><b>b</b></i></sup></a></span> <span class="reference-text"><cite id="CITEREFNemirovsky_and_Ben-Tal2023" class="citation web cs1">Nemirovsky and Ben-Tal (2023). <a rel="nofollow" class="external text" href="http://www2.isye.gatech.edu/~nemirovs/OPTIIILN2023Spring.pdf">"Optimization III: Convex Optimization"</a> <span class="cs1-format">(PDF)</span>.</cite></span>
</li>
</ol></div></div>
<div class="mw-heading mw-heading2"><h2 id="Further_reading">Further reading</h2></div>
<ul><li>Avriel, Mordecai (2003). <i>Nonlinear Programming: Analysis and Methods.</i> Dover Publishing. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a>&nbsp;<bdi>0-486-43227-0</bdi>.</li>
<li>Bazaraa, Mokhtar S. and Shetty, C. M. (1979). <i>Nonlinear programming. Theory and algorithms.</i> John Wiley &amp; Sons. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a>&nbsp;<bdi>0-471-78610-1</bdi>.</li>
<li><cite id="CITEREFBonnansGilbertLemaréchalSagastizábal2006" class="citation book cs1">Bonnans, J.&nbsp;Frédéric; Gilbert, J.&nbsp;Charles; <a href="Claude_Lemar%C3%A9chal" title="Claude Lemaréchal">Lemaréchal, Claude</a>; <a href="Claudia_Sagastiz%C3%A1bal" title="Claudia Sagastizábal">Sagastizábal, Claudia&nbsp;A.</a> (2006). <a rel="nofollow" class="external text" href="https://www.springer.com/mathematics/applications/book/978-3-540-35445-1"><i>Numerical optimization: Theoretical and practical aspects</i></a>. Universitext (Second revised ed. of translation of 1997 French&nbsp;ed.). Berlin: Springer-Verlag. pp.&nbsp;xiv+490. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2F978-3-540-35447-5">10.1007/978-3-540-35447-5</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a>&nbsp;<bdi>3-540-35445-X</bdi>. <a href="MR_(identifier)" class="mw-redirect" title="MR (identifier)">MR</a>&nbsp;<a rel="nofollow" class="external text" href="https://mathscinet.ams.org/mathscinet-getitem?mr=2265882">2265882</a>.</cite></li>
<li><cite id="CITEREFLuenbergerYe2008" class="citation book cs1"><a href="David_G._Luenberger" class="mw-redirect" title="David G. Luenberger">Luenberger, David G.</a>; <a href="Yinyu_Ye" title="Yinyu Ye">Ye, Yinyu</a> (2008). <i>Linear and nonlinear programming</i>. International Series in Operations Research &amp; Management Science. Vol.&nbsp;116 (Third&nbsp;ed.). New York: Springer. pp.&nbsp;xiv+546. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a>&nbsp;<bdi>978-0-387-74502-2</bdi>. <a href="MR_(identifier)" class="mw-redirect" title="MR (identifier)">MR</a>&nbsp;<a rel="nofollow" class="external text" href="https://mathscinet.ams.org/mathscinet-getitem?mr=2423726">2423726</a>.</cite></li>
<li>Nocedal, Jorge and Wright, Stephen J. (1999). <i>Numerical Optimization.</i> Springer. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a>&nbsp;<bdi>0-387-98793-2</bdi>.</li>
<li><a href="Jan_Brinkhuis" title="Jan Brinkhuis">Jan Brinkhuis</a> and Vladimir Tikhomirov, <i>Optimization: Insights and Applications</i>, 2005, Princeton University Press</li></ul>
<div class="mw-heading mw-heading2"><h2 id="External_links">External links</h2></div>
<ul><li><a rel="nofollow" class="external text" href="http://glossary.computing.society.informs.org/">Mathematical Programming Glossary</a></li></ul>
<div class="navbox-styles"><style data-mw-deduplicate="TemplateStyles:r1129693374">
/* start https://en.wikipedia.org/ */


.mw-parser-output .hlist dl,.mw-parser-output .hlist ol,.mw-parser-output .hlist ul{margin:0;padding:0}.mw-parser-output .hlist dd,.mw-parser-output .hlist dt,.mw-parser-output .hlist li{margin:0;display:inline}.mw-parser-output .hlist.inline,.mw-parser-output .hlist.inline dl,.mw-parser-output .hlist.inline ol,.mw-parser-output .hlist.inline ul,.mw-parser-output .hlist dl dl,.mw-parser-output .hlist dl ol,.mw-parser-output .hlist dl ul,.mw-parser-output .hlist ol dl,.mw-parser-output .hlist ol ol,.mw-parser-output .hlist ol ul,.mw-parser-output .hlist ul dl,.mw-parser-output .hlist ul ol,.mw-parser-output .hlist ul ul{display:inline}.mw-parser-output .hlist .mw-empty-li{display:none}.mw-parser-output .hlist dt::after{content:": "}.mw-parser-output .hlist dd::after,.mw-parser-output .hlist li::after{content:" · ";font-weight:bold}.mw-parser-output .hlist dd:last-child::after,.mw-parser-output .hlist dt:last-child::after,.mw-parser-output .hlist li:last-child::after{content:none}.mw-parser-output .hlist dd dd:first-child::before,.mw-parser-output .hlist dd dt:first-child::before,.mw-parser-output .hlist dd li:first-child::before,.mw-parser-output .hlist dt dd:first-child::before,.mw-parser-output .hlist dt dt:first-child::before,.mw-parser-output .hlist dt li:first-child::before,.mw-parser-output .hlist li dd:first-child::before,.mw-parser-output .hlist li dt:first-child::before,.mw-parser-output .hlist li li:first-child::before{content:" (";font-weight:normal}.mw-parser-output .hlist dd dd:last-child::after,.mw-parser-output .hlist dd dt:last-child::after,.mw-parser-output .hlist dd li:last-child::after,.mw-parser-output .hlist dt dd:last-child::after,.mw-parser-output .hlist dt dt:last-child::after,.mw-parser-output .hlist dt li:last-child::after,.mw-parser-output .hlist li dd:last-child::after,.mw-parser-output .hlist li dt:last-child::after,.mw-parser-output .hlist li li:last-child::after{content:")";font-weight:normal}.mw-parser-output .hlist ol{counter-reset:listitem}.mw-parser-output .hlist ol>li{counter-increment:listitem}.mw-parser-output .hlist ol>li::before{content:" "counter(listitem)"\a0 "}.mw-parser-output .hlist dd ol>li:first-child::before,.mw-parser-output .hlist dt ol>li:first-child::before,.mw-parser-output .hlist li ol>li:first-child::before{content:" ("counter(listitem)"\a0 "}


/* end https://en.wikipedia.org/ */
</style><style data-mw-deduplicate="TemplateStyles:r1236075235">
/* start https://en.wikipedia.org/ */


.mw-parser-output .navbox{box-sizing:border-box;border:1px solid #a2a9b1;width:100%;clear:both;font-size:88%;text-align:center;padding:1px;margin:1em auto 0}.mw-parser-output .navbox .navbox{margin-top:0}.mw-parser-output .navbox+.navbox,.mw-parser-output .navbox+.navbox-styles+.navbox{margin-top:-1px}.mw-parser-output .navbox-inner,.mw-parser-output .navbox-subgroup{width:100%}.mw-parser-output .navbox-group,.mw-parser-output .navbox-title,.mw-parser-output .navbox-abovebelow{padding:0.25em 1em;line-height:1.5em;text-align:center}.mw-parser-output .navbox-group{white-space:nowrap;text-align:right}.mw-parser-output .navbox,.mw-parser-output .navbox-subgroup{background-color:#fdfdfd}.mw-parser-output .navbox-list{line-height:1.5em;border-color:#fdfdfd}.mw-parser-output .navbox-list-with-group{text-align:left;border-left-width:2px;border-left-style:solid}.mw-parser-output tr+tr>.navbox-abovebelow,.mw-parser-output tr+tr>.navbox-group,.mw-parser-output tr+tr>.navbox-image,.mw-parser-output tr+tr>.navbox-list{border-top:2px solid #fdfdfd}.mw-parser-output .navbox-title{background-color:#ccf}.mw-parser-output .navbox-abovebelow,.mw-parser-output .navbox-group,.mw-parser-output .navbox-subgroup .navbox-title{background-color:#ddf}.mw-parser-output .navbox-subgroup .navbox-group,.mw-parser-output .navbox-subgroup .navbox-abovebelow{background-color:#e6e6ff}.mw-parser-output .navbox-even{background-color:#f7f7f7}.mw-parser-output .navbox-odd{background-color:transparent}.mw-parser-output .navbox .hlist td dl,.mw-parser-output .navbox .hlist td ol,.mw-parser-output .navbox .hlist td ul,.mw-parser-output .navbox td.hlist dl,.mw-parser-output .navbox td.hlist ol,.mw-parser-output .navbox td.hlist ul{padding:0.125em 0}.mw-parser-output .navbox .navbar{display:block;font-size:100%}.mw-parser-output .navbox-title .navbar{float:left;text-align:left;margin-right:0.5em}body.skin--responsive .mw-parser-output .navbox-image img{max-width:none!important}@media print{body.ns-0 .mw-parser-output .navbox{display:none!important}}


/* end https://en.wikipedia.org/ */
</style></div><div role="navigation" class="navbox" aria-labelledby="Optimization:_Algorithms,_methods,_and_heuristics381" style="padding:3px"><table class="nowraplinks hlist mw-collapsible uncollapsed navbox-inner" style="border-spacing:0;background:transparent;color:inherit"><tbody><tr><th scope="col" class="navbox-title" colspan="3"><style data-mw-deduplicate="TemplateStyles:r1239400231">
/* start https://en.wikipedia.org/ */


.mw-parser-output .navbar{display:inline;font-size:88%;font-weight:normal}.mw-parser-output .navbar-collapse{float:left;text-align:left}.mw-parser-output .navbar-boxtext{word-spacing:0}.mw-parser-output .navbar ul{display:inline-block;white-space:nowrap;line-height:inherit}.mw-parser-output .navbar-brackets::before{margin-right:-0.125em;content:"[ "}.mw-parser-output .navbar-brackets::after{margin-left:-0.125em;content:" ]"}.mw-parser-output .navbar li{word-spacing:-0.125em}.mw-parser-output .navbar a>span,.mw-parser-output .navbar a>abbr{text-decoration:inherit}.mw-parser-output .navbar-mini abbr{font-variant:small-caps;border-bottom:none;text-decoration:none;cursor:inherit}.mw-parser-output .navbar-ct-full{font-size:114%;margin:0 7em}.mw-parser-output .navbar-ct-mini{font-size:114%;margin:0 4em}html.skin-theme-clientpref-night .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}@media(prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}}@media print{.mw-parser-output .navbar{display:none!important}}


/* end https://en.wikipedia.org/ */
</style><div id="Optimization:_Algorithms,_methods,_and_heuristics381" style="font-size:114%;margin:0 4em"><a href="Mathematical_optimization" title="Mathematical optimization">Optimization</a>: <a href="Optimization_algorithm" class="mw-redirect" title="Optimization algorithm">Algorithms</a>, <a href="Iterative_method" title="Iterative method">methods</a>, and <a href="Heuristic_algorithm" class="mw-redirect" title="Heuristic algorithm">heuristics</a></div></th></tr><tr><td colspan="2" class="navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks mw-collapsible uncollapsed navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><div id="Unconstrained_nonlinear381" style="font-size:114%;margin:0 4em"></div></th></tr><tr><td colspan="2" class="navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Function_(mathematics)" title="Function (mathematics)">Functions</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Golden-section_search" title="Golden-section search">Golden-section search</a></li>
<li><a href="Powell's_method" title="Powell's method">Powell's method</a></li>
<li><a href="Line_search" title="Line search">Line search</a></li>
<li><a href="Nelder%E2%80%93Mead_method" title="Nelder–Mead method">Nelder–Mead method</a></li>
<li><a href="Successive_parabolic_interpolation" title="Successive parabolic interpolation">Successive parabolic interpolation</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Gradient" title="Gradient">Gradients</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Local_convergence" title="Local convergence">Convergence</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Trust_region" title="Trust region">Trust region</a></li>
<li><a href="Wolfe_conditions" title="Wolfe conditions">Wolfe conditions</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Quasi-Newton_method" title="Quasi-Newton method">Quasi–Newton</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Berndt%E2%80%93Hall%E2%80%93Hall%E2%80%93Hausman_algorithm" title="Berndt–Hall–Hall–Hausman algorithm">Berndt–Hall–Hall–Hausman</a></li>
<li><a href="Broyden%E2%80%93Fletcher%E2%80%93Goldfarb%E2%80%93Shanno_algorithm" title="Broyden–Fletcher–Goldfarb–Shanno algorithm">Broyden–Fletcher–Goldfarb–Shanno</a> and <a href="Limited-memory_BFGS" title="Limited-memory BFGS">L-BFGS</a></li>
<li><a href="Davidon%E2%80%93Fletcher%E2%80%93Powell_formula" title="Davidon–Fletcher–Powell formula">Davidon–Fletcher–Powell</a></li>
<li><a href="Symmetric_rank-one" title="Symmetric rank-one">Symmetric rank-one (SR1)</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Iterative_method" title="Iterative method">Other methods</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Nonlinear_conjugate_gradient_method" title="Nonlinear conjugate gradient method">Conjugate gradient</a></li>
<li><a href="Gauss%E2%80%93Newton_algorithm" title="Gauss–Newton algorithm">Gauss–Newton</a></li>
<li><a href="Gradient_descent" title="Gradient descent">Gradient</a></li>
<li><a href="Mirror_descent" title="Mirror descent">Mirror</a></li>
<li><a href="Levenberg%E2%80%93Marquardt_algorithm" title="Levenberg–Marquardt algorithm">Levenberg–Marquardt</a></li>
<li><a href="Powell's_dog_leg_method" title="Powell's dog leg method">Powell's dog leg method</a></li>
<li><a href="Truncated_Newton_method" title="Truncated Newton method">Truncated Newton</a></li></ul>
</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Hessian_matrix" title="Hessian matrix">Hessians</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Newton's_method_in_optimization" title="Newton's method in optimization">Newton's method</a></li></ul>
</div></td></tr></tbody></table><div></div></td></tr></tbody></table><div></div></td><td class="noviewer navbox-image" rowspan="5" style="width:1px;padding:0 0 0 2px"><div></div></td></tr><tr><td colspan="2" class="navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks mw-collapsible uncollapsed navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><div id="Constrained_nonlinear381" style="font-size:114%;margin:0 4em"></div></th></tr><tr><td colspan="2" class="navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">General</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Barrier_function" title="Barrier function">Barrier methods</a></li>
<li><a href="Penalty_method" title="Penalty method">Penalty methods</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Differentiable</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Augmented_Lagrangian_method" title="Augmented Lagrangian method">Augmented Lagrangian methods</a></li>
<li><a href="Sequential_quadratic_programming" title="Sequential quadratic programming">Sequential quadratic programming</a></li>
<li><a href="Successive_linear_programming" title="Successive linear programming">Successive linear programming</a></li></ul>
</div></td></tr></tbody></table><div></div></td></tr></tbody></table><div></div></td></tr><tr><td colspan="2" class="navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks mw-collapsible mw-collapsed navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><div id="Convex_optimization381" style="font-size:114%;margin:0 4em"><a href="Convex_optimization" title="Convex optimization">Convex optimization</a></div></th></tr><tr><td colspan="2" class="navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Convex_minimization" class="mw-redirect" title="Convex minimization">Convex<br> minimization</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Cutting-plane_method" title="Cutting-plane method">Cutting-plane method</a></li>
<li><a href="Frank%E2%80%93Wolfe_algorithm" title="Frank–Wolfe algorithm">Reduced gradient (Frank–Wolfe)</a></li>
<li><a href="Subgradient_method" title="Subgradient method">Subgradient method</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Linear_programming" title="Linear programming">Linear</a> and<br><a href="Quadratic_programming" title="Quadratic programming">quadratic</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Linear_programming#Interior_point" title="Linear programming">Interior point</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Affine_scaling" title="Affine scaling">Affine scaling</a></li>
<li><a href="Ellipsoid_method" title="Ellipsoid method">Ellipsoid algorithm of Khachiyan</a></li>
<li><a href="Karmarkar's_algorithm" title="Karmarkar's algorithm">Projective algorithm of Karmarkar</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Matroid" title="Matroid">Basis-</a><a href="Exchange_algorithm" class="mw-redirect" title="Exchange algorithm">exchange</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Simplex_algorithm" title="Simplex algorithm">Simplex algorithm of Dantzig</a></li>
<li><a href="Revised_simplex_method" title="Revised simplex method">Revised simplex algorithm</a></li>
<li><a href="Criss-cross_algorithm" title="Criss-cross algorithm">Criss-cross algorithm</a></li>
<li><a href="Lemke's_algorithm" title="Lemke's algorithm">Principal pivoting algorithm of Lemke</a></li>
<li><a href="Active-set_method" title="Active-set method">Active-set method</a></li></ul>
</div></td></tr></tbody></table><div></div></td></tr></tbody></table><div></div></td></tr></tbody></table><div></div></td></tr><tr><td colspan="2" class="navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks mw-collapsible mw-collapsed navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><div id="Combinatorial381" style="font-size:114%;margin:0 4em"><a href="Combinatorial_optimization" title="Combinatorial optimization">Combinatorial</a></div></th></tr><tr><td colspan="2" class="navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">Paradigms</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Approximation_algorithm" title="Approximation algorithm">Approximation algorithm</a></li>
<li><a href="Dynamic_programming" title="Dynamic programming">Dynamic programming</a></li>
<li><a href="Greedy_algorithm" title="Greedy algorithm">Greedy algorithm</a></li>
<li><a href="Integer_programming" title="Integer programming">Integer programming</a>
<ul><li><a href="Branch_and_bound" title="Branch and bound">Branch and bound</a>/<a href="Branch_and_cut" title="Branch and cut">cut</a></li></ul></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Graph_algorithm" class="mw-redirect" title="Graph algorithm">Graph<br> algorithms</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th id="Minimum_spanning_tree52" scope="row" class="navbox-group" style="width:1%"><a href="Minimum_spanning_tree" title="Minimum spanning tree">Minimum<br> spanning tree</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Bor%C5%AFvka's_algorithm" title="Borůvka's algorithm">Borůvka</a></li>
<li><a href="Prim's_algorithm" title="Prim's algorithm">Prim</a></li>
<li><a href="Kruskal's_algorithm" title="Kruskal's algorithm">Kruskal</a></li></ul>
</div></td></tr></tbody></table><div>
</div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th id="Shortest_path39" scope="row" class="navbox-group" style="width:1%"><a href="Shortest_path_problem" title="Shortest path problem">Shortest path</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Bellman%E2%80%93Ford_algorithm" title="Bellman–Ford algorithm">Bellman–Ford</a>
<ul><li><a href="Shortest_Path_Faster_Algorithm" class="mw-redirect" title="Shortest Path Faster Algorithm">SPFA</a></li></ul></li>
<li><a href="Dijkstra's_algorithm" title="Dijkstra's algorithm">Dijkstra</a></li>
<li><a href="Floyd%E2%80%93Warshall_algorithm" title="Floyd–Warshall algorithm">Floyd–Warshall</a></li></ul>
</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Flow_network" title="Flow network">Network flows</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Dinic's_algorithm" title="Dinic's algorithm">Dinic</a></li>
<li><a href="Edmonds%E2%80%93Karp_algorithm" title="Edmonds–Karp algorithm">Edmonds–Karp</a></li>
<li><a href="Ford%E2%80%93Fulkerson_algorithm" title="Ford–Fulkerson algorithm">Ford–Fulkerson</a></li>
<li><a href="Push%E2%80%93relabel_maximum_flow_algorithm" title="Push–relabel maximum flow algorithm">Push–relabel maximum flow</a></li></ul>
</div></td></tr></tbody></table><div></div></td></tr></tbody></table><div></div></td></tr><tr><td colspan="2" class="navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks mw-collapsible mw-collapsed navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><div id="Metaheuristics381" style="font-size:114%;margin:0 4em"><a href="Metaheuristic" title="Metaheuristic">Metaheuristics</a></div></th></tr><tr><td colspan="2" class="navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Evolutionary_algorithm" title="Evolutionary algorithm">Evolutionary algorithm</a></li>
<li><a href="Hill_climbing" title="Hill climbing">Hill climbing</a></li>
<li><a href="Local_search_(optimization)" title="Local search (optimization)">Local search</a></li>
<li><a href="Parallel_metaheuristic" title="Parallel metaheuristic">Parallel metaheuristics</a></li>
<li><a href="Simulated_annealing" title="Simulated annealing">Simulated annealing</a></li>
<li><a href="Spiral_optimization_algorithm" title="Spiral optimization algorithm">Spiral optimization algorithm</a></li>
<li><a href="Tabu_search" title="Tabu search">Tabu search</a></li></ul>
</div></td></tr></tbody></table><div></div></td></tr><tr><td class="navbox-abovebelow" colspan="3"><div>
<ul><li><a href="Comparison_of_optimization_software" title="Comparison of optimization software">Software</a></li></ul>
</div></td></tr></tbody></table></div>
<div class="navbox-styles"><style data-mw-deduplicate="TemplateStyles:r1038841319">
/* start https://en.wikipedia.org/ */


.mw-parser-output .tooltip-dotted{border-bottom:1px dotted;cursor:help}


/* end https://en.wikipedia.org/ */
</style></div><div role="navigation" class="navbox authority-control" aria-labelledby="Authority_control_databases_frameless&amp;#124;text-top&amp;#124;10px&amp;#124;alt=Edit_this_at_Wikidata&amp;#124;link=https&amp;#58;//www.wikidata.org/wiki/Q769909#identifiers&amp;#124;class=noprint&amp;#124;Edit_this_at_Wikidata1552" style="padding:3px"><table class="nowraplinks hlist mw-collapsible autocollapse navbox-inner" style="border-spacing:0;background:transparent;color:inherit"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><div id="Authority_control_databases_frameless&amp;#124;text-top&amp;#124;10px&amp;#124;alt=Edit_this_at_Wikidata&amp;#124;link=https&amp;#58;//www.wikidata.org/wiki/Q769909#identifiers&amp;#124;class=noprint&amp;#124;Edit_this_at_Wikidata1552" style="font-size:114%;margin:0 4em">Authority control databases </div></th></tr><tr><th scope="row" class="navbox-group" style="width:1%">National</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"><ul><li><span class="uid"><span class="rt-commentedText tooltip tooltip-dotted" title="Nonlinear programming"><a rel="nofollow" class="external text" href="https://id.loc.gov/authorities/sh85092331">United States</a></span></span></li><li><span class="uid"><span class="rt-commentedText tooltip tooltip-dotted" title="Programmation non linéaire"><a rel="nofollow" class="external text" href="https://catalogue.bnf.fr/ark:/12148/cb122677537">France</a></span></span></li><li><span class="uid"><span class="rt-commentedText tooltip tooltip-dotted" title="Programmation non linéaire"><a rel="nofollow" class="external text" href="https://data.bnf.fr/ark:/12148/cb122677537">BnF data</a></span></span></li><li><span class="uid"><span class="rt-commentedText tooltip tooltip-dotted" title="nelineární programování"><a rel="nofollow" class="external text" href="https://aleph.nkp.cz/F/?func=find-c&amp;local_base=aut&amp;ccl_term=ica=ph293203&amp;CON_LNG=ENG">Czech Republic</a></span></span></li><li><span class="uid"><a rel="nofollow" class="external text" href="https://www.nli.org.il/en/authorities/987007533977205171">Israel</a></span></li></ul></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Other</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em"><ul><li><span class="uid"><a rel="nofollow" class="external text" href="https://lux.collections.yale.edu/view/concept/06087ac1-98b3-4c29-92c6-fa55efa245a7">Yale LUX</a></span></li></ul></div></td></tr></tbody></table></div></div><!--htdig_noindex--><div><div class="zim-footer">
This article is issued from <a class="external text" title="Last edited on 2024-08-15" href="https://en.wikipedia.org/wiki/?title=Nonlinear_programming&amp;oldid=1240445515">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.
</div>
</div><!--/htdig_noindex--></div>
</div>
</main>
</div>
</div>
</div>

</body></html>